A Multithreaded Message Passing Interface (MPI) Architecture: Performance and Program Issues

Device Interface (ADI ) Channel Device Interface Low-level device MPI Implementation MPI Application Communication Device Figure 1. MPICH layered software architecture The deficiencies of the MPICH architecture, such as inefficient multi-fabric communication and non-thread-safety are rooted in the ADI and Device layers. In order to make further discussion specific, we present the ADI and CDI in more detail below. 2.1 Abstract Device Interface (ADI) Message passing in MPI is organized in the following way. When users call MPI_Send or MPI_Recv, a data structure that contains an instantaneous description of the data transfer operation is created (we refer to it as transfer description later in the paper). The data structure is reference by an opaque pointer called handle. The handles to transfers are placed in send and receive queues that are used in order to ensure the correct order of transfer completion. The